Development Of A Cloud Computing Application For Water Resources Modelling And Optimization Based On Open Source Software

Cloud computing is the latest advancement in Information and Communication Technology (ICT) that provides computing as a service or delivers computation, software, data access, storage service without end-user knowledge of the physical location and system configuration. Cloud computing, service oriented architecture and web geographic information systems are new technologies for development of the cloud computing application for water resources modelling and optimization. The cloud application is deployed and tested in a distributed computer environment running on three virtual machines (VMs). The cloud application has five web services for: (1) spatial data infrastructure – 1 (SDI), (2) SDI – 2, (3) support for water resources modelling (4) water resources optimization and 5) user authentication. The cloud application is developed using several programming languages (PHP, Ajax, Java, and JavaScript), libraries (OpenLayers and JQuery) and open-source software components (GeoServer, PostgreSQL and PostGIS) and OGC standards (WMS, WFS and WFT-T). The web services for support of water resources modelling and user authentication are deployed on Amazon Web Services and are communicating using WFS with the two SDI web services. The two SDI web services are working on the two separate VMs providing geospatial data and services. The fourth web service is deployed on a separate VM because of the expected large computational requirements. The cloud application is scalable, interoperable, creates a real time multi-user collaboration platform. All code and components used are open source. The cloud application was tested with concurrent multiple users. The performance, security and utilization of the distributed computer environment are monitored and analysed together with the users’ experience and satisfaction. The applicability of the presented solution and its future are elaborated

Committees Of Specialized Conceptual Hydrological Models: Comparative Study

Committee modelling approach is skillful prediction in the domain of hydrological modelling that allows explicitly to derive predictive model outputs. In this approach, the different individual models are optimally combined. Generally if a single hydrological model or the model calibrated by the single aggregated objective function it is hard to capture all facets of a complex process and to present the best possible model outputs. This model could be either capable for high flows or for low flows or not for both cases hence more flexible modelling architectures are required. Here the possibilities is building several specialized models each of which is responsible for a particular sub-process (high flows or low flows), and combining them using dynamic weights – thus forming a committee model. In this study we compare two different types of committee models: (i) the combine model based on fuzzy memberships function (Kayastha et al. 2013, Fenicia et al. 2007) and (ii) the combine model based on weights that calculated from hydrological states (Oudin et al. 2006). Before combining the models the individual hydrological models are calibrated by Adaptive Cluster Covering Algorithm (Solomatine 1999) for high and low flows with (different) suitable objective functions. The committee model based on fuzzy memberships does not generate additional water in the system (preserves water balance), however there is no guarantee for this in case of committees based on hydrological states. The relative performances of the two different committee models and their characteristics are illustrated, with an application to HBV hydrological models in Bagmati catchment in Nepal

Effect Of Different Hydrological Model Structures On The Assimilation Of Distributed Uncertain Observations

The reliable evaluation of the flood forecasting is a crucial problem for assessing flood risk and consequent damages. Different hydrological models (distributed, semi-distributed or lumped) have been proposed in order to deal with this issue. The choice of the proper model structure has been investigated by many authors and it is one of the main sources of uncertainty for a correct evaluation of the outflow hydrograph. In addition, the recent increasing of data availability makes possible to update hydrological models as response of real-time observations. For these reasons, the aim of this work it is to evaluate the effect of different structure of a semi-distributed hydrological model in the assimilation of distributed uncertain discharge observations. The study was applied to the Bacchiglione catchment, located in Italy. The first methodological step was to divide the basin in different sub-basins according to topographic characteristics. Secondly, two different structures of the semi-distributed hydrological model were implemented in order to estimate the outflow hydrograph. Then, synthetic observations of uncertain value of discharge were generated, as a function of the observed and simulated value of flow at the basin outlet, and assimilated in the semi-distributed models using a Kalman Filter. Finally, different spatial patterns of sensors location were assumed to update the model state as response of the uncertain discharge observations. The results of this work pointed out that, overall, the assimilation of uncertain observations can improve the hydrologic model performance. In particular, it was found that the model structure is an important factor, of difficult characterization, since can induce different forecasts in terms of outflow discharge. This study is partly supported by the FP7 EU Project WeSenseIt

Multiobjective direct policy search using physically based operating rules in multireservoir systems

supplemental_data_wr.1943-5452.0001159_ritter.pdf (492 KB)This study explores the ways to introduce physical interpretability into the process of optimizing operating rules for multireservoir systems with multiple objectives. Prior studies applied the concept of direct policy search (DPS), in which the release policy is expressed as a set of parameterized functions (e.g., neural networks) that are optimized by simulating the performance of different parameter value combinations over a testing period. The problem with this approach is that the operators generally avoid adopting such artificial black-box functions for the direct real-time control of their systems, preferring simpler tools with a clear connection to the system physics. This study addresses this mismatch by replacing the black-box functions in DPS with physically based parameterized operating rules, for example by directly using target levels in dams as decision variables. This leads to results that are physically interpretable and may be more acceptable to operators. The methodology proposed in this work is applied to a network of five reservoirs and four power plants in the Nechi catchment in Colombia, with four interests involved: average energy generation, firm energy generation, flood hazard, and flow regime alteration. The release policy is expressed depending on only 12 parameters, which significantly reduces the computational complexity compared to existing approaches of multiobjective DPS. The resulting four-dimensional Pareto-approximate set offers a variety of operational strategies from which operators may choose one that corresponds best to their preferences. For demonstration purposes, one particular optimized policy is selected and its parameter values are analyzed to illustrate how the physically based operating rules can be directly interpreted by the operators.Peer ReviewedPreprin

Prediction Of Hydrological Models’ Uncertainty By A Committee Of Machine Learning-Models

This study presents an approach to combine uncertainties of the hydrological model outputs predicted from a number of machine learning models. The machine learning based uncertainty prediction approach is very useful for estimation of hydrological models\u27 uncertainty in particular hydro-metrological situation in real-time application [1]. In this approach the hydrological model realizations from Monte Carlo simulations are used to build different machine learning uncertainty models to predict uncertainty (quantiles of pdf) of the a deterministic output from hydrological model . Uncertainty models are trained using antecedent precipitation and streamflows as inputs. The trained models are then employed to predict the model output uncertainty which is specific for the new input data. We used three machine learning models namely artificial neural networks, model tree, locally weighted regression to predict output uncertainties. These three models produce similar verification results, which can be improved by merging their outputs dynamically. We propose an approach to form a committee of the three models to combine their outputs. The approach is applied to estimate uncertainty of streamflows simulation from a conceptual hydrological model in the Brue catchment in UK and the Bagmati catchment in Nepal. The verification results show that merged output is better than an individual model output. [1] D. L. Shrestha, N. Kayastha, and D. P. Solomatine, and R. Price. Encapsulation of parameteric uncertainty statistics by various predictive machine learning models: MLUE method, Journal of Hydroinformatic, in press, 2013

Assimilation Of Heterogeneous Uncertain Data, Having Different Observational Errors, In Hydrological Models

Accurate real-time forecasting of river water level is an important issue that has to be addressed in order to prevent and mitigate water-related risk. To this end, data assimilation methods have been used to improve the forecasts ability of water model merging observations coming from stations and model simulations. As a consequence of the increasing availability of dynamic and cheap sensors, having variable life-span, space and temporal coverage, the citizens are becoming an active part in information capturing, evaluation and communication. On the other hand, it is difficult to assess the uncertain related to the observation coming from such sensors. The main objective of this work is to evaluate the influence of the observational error in the proposed assimilation methodologies used to update the hydrological model as response of dynamic observations of water discharge. We tested the developed approaches on a test study area - the Brue catchment, located in the South West of England, UK. Two different filtering approaches, Ensemble Kalman filter and Particle filter, were applied to the semi-distributed hydrological model. Discharge observations were synthetically generated as a function of the observed and simulated value of flow at the basin outlet. Different types of observational error were introduced assuming diverse sets of probability distributions, first and second order moments. The results of this work show how the assimilation of dynamic observations, in time and space, can improve the hydrologic model performance with a better forecast of flood events. It was found that the choice of the appropriate observational error, of difficult characterization, and type of filtering approach affects the model accuracy. This study is partly supported by the FP7 EU Project WeSenseIt

Precipitation Sensor Network Optimal Design Using Time-Space Varying Correlation Structure

Design of optimal precipitation sensor networks is a common topic in hydrological literature, however this is still an open problem due to lack of understanding of some spatially variable processes, and assumptions that often cannot be verified. Among these assumptions lies the homoscedasticity of precipitation fields, common in hydrological practice. To overcome this, it is proposed a local intensity-variant covariance structure, which in the broad extent, provides a fully updated correlation structure as long as new data are coming into the system. These considerations of intensity-variant correlation structure will be tested in the design of a precipitation sensor network for a case study, improving the estimation of precipitation fields, and thus, reducing the input uncertainty in hydrological models, especially in the scope of rainfall-runoff models

Model Calibration using ANN-ACCO Optimization Method

In order to properly simulate the natural phenomena using numerical model, model parameters have to be estimated by an appropriate manner. Here, new approach using ACCO and artificial neural network is proposed for the calibration of numerical simulation model. ANN works as an error estimator in this proposed model. From the comparison of results with ACCO, although the number of function evaluation time is larger than that of ACCO, it is shown that the optimization by ANN-ACCO is reasonably carried out with better accuracy and stability. Model calibration was also successfully established by ANN-ACCO, then the some degree of its applicability to model calibrations were shown

ВОЗМОЖНОСТИ КРАТКОСРОЧНОГО ПРОГНОЗИРОВАНИЯ СТОКА МАЛОЙ РЕКИ С ИСПОЛЬЗОВАНИЕМ МЕТОДОВ МАШИННОГО ОБУЧЕНИЯ

The paper addresses prospects for short-term (from 1 to 7 days) forecasting of river streamflow runoff based on several machine learning methods: multiple linear regression (LM) model, a multilayer perceptron (MLP) artificial neural network, and a recurrent artificial neural network with long short-term memory (LSTM). Methods for expanding the set of predictors for model construction are proposed, and the possibility of random shuffling of the time-series of predictors for model calibration and verification are assessed. The object of the study is the small river of Central Russia – river Protva (Spas-Zagorie gauge). Current and lagged values of streamflow discharge at the gauge and daily precipitation at local weather stations are used as predictors for the model, as well as moisture index and evaporation rate. The obtained results show the possibility of constructing an effective operational forecasting system for short-term runoff forecasting. The study revealed the applicability of artificial neural network models, acceptable for operational practice, using all available hydrometeorological information on the catchment, as they showed the most stable results at all lead times from 1 to 7 days. In contrast to the linear model, which efficiency decreases after lead time of more than 3 days, the artificial neural networks models have higher forecast efficiency up to 7 days. The results obtained are robust for all phases of the water regime, both spring floods and summer floods. The software implementation of the models is made on the basis of open software libraries in the Python language, which makes it possible to widely use the methods for scientific research and applied problems.В статье исследуются возможности краткосрочного (от 1 до 7 суток) прогнозирования расходов воды на основе нескольких методов машинного обучения: модели множественной линейной регрессии, искусственной нейронной сети по типу многослойного перцептрона и рекуррентной искусственной нейронной сети с долгосрочной кратковременной памятью. Предлагаются методы расширения набора предикторов для построения моделей и исследуется возможность случайного перемешивания хронологического ряда предикторов для калибровки и верификации моделей как повышающая устойчивость результатов прогноза. В качестве объекта исследования используется малая река Средней полосы России – река Протва (гидрометрический пост Спас-Загорье). В качестве предикторов используются расходы воды на посту и суточные суммы осадков на трех ближайших метеостанциях в текущий момент времени (сутки) и со сдвигом назад до 7 суток, а также индекс увлажнения бассейна и характеристики температуры воздуха и испарения. На конкретном примере показана возможность построения эффективной оперативной прогностической системы для краткосрочного прогнозирования стока. Исследование выявило приемлемую для оперативной практики применимость моделей искусственных нейронных сетей, использующих всю доступную гидрометеорологическую информацию на водосборе, как показавших наиболее устойчивые результаты на всех заблаговременностях от 1 до 7 суток. Так, в отличие от линейной прогностической модели, эффективность которой снижается на заблаговременностях более 3 суток, модели искусственных нейронных сетей показали высокую эффективность прогноза до 7 суток. Полученные результаты устойчивы для всех фаз водного режима, как весеннего половодья, так и летних паводков. Программная реализация моделей выполнена на основании открытых программных библиотек на языке Python, что показывает возможность широкого использования описанных методик для научных исследований и прикладных задач

Automatic Calibration of Water Quality Simulation Model Using Adaptive Clustre Covering Method

Model calibration is important procedures for confirming accuracy of predicted results and validity of modeling. In this study, adaptive cluster covering method (ACCO method) was used for the automatic calibration of water quality simulation model, and applicability of this method was discussed. Although the test of model calibration was executed for limited datasets, calibration was processed well by this method. The calibrated model simulates the tendency of water quality changes. Accordingly, it can be noted that this optimization method is applicable to automatic calibration of simulation model